Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,524 questions
0 answers
Changing the executor count in the Spark job definition configuration in Azure Synapse Analytics is not being reflected in the job
Hello All, Since my job was running for a long time, I did some analysis and decided to change the executor count to reduce the job running duration. My previous executor count was 4, and I tried to scale it up to 6. But the issue is, I could only…
asked 2024-06-07T07:02:11.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 13%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
Saravana Gopal
25
Reputation points
commented 2024-06-09T06:36:52.0766667+00:00
Saravana Gopal
25
Reputation points
1 answer
Self-hosted machine unable to access Data Lake Storage account when running a pipeline using Synapse
Hello - I need your help again: Here's the story: I have azure synapse workspace I have created managed private end point created for ADLS - working fine I have create private endpoint created for ADLS - working fine ADLS have set the public access…
asked 2024-06-07T08:58:57.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 88%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXM%3C/text%3E%3C/svg%3E)
Xhevahir Mehalla
80
Reputation points
answered 2024-06-08T14:26:42.6633333+00:00
Amira Bedhiafi
17,616
Reputation points
0 answers
Error publishing notebook.
I am trying to publish my notebook. I get the following error : CreateOrUpdateNotebook failed: BadRequest. The operation failed because the entity “notebookA” is being renamed to “notebookB”. A retry of rename from exact source to target may work”. It…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 14.000000000000002%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECT%3C/text%3E%3C/svg%3E)
1 answer
add network rules and allow only inbound traffic from synapse and remove public access.
Hi All, We use Synapse spark jobs which connects to the data sources. Wanted to check if there is Service Tag available in Synapse so that we can add network rules and allow only inbound traffic from synapse and remove public access. Please let us…
asked 2024-06-06T20:49:27.5366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 49%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGM%3C/text%3E%3C/svg%3E)
Gustavo Mercado (LTIMINDTREE LIMITED)
0
Reputation points Microsoft Vendor
edited an answer 2024-06-07T17:54:51.2+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 48%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EH%3C/text%3E%3C/svg%3E)
Harishga
4,640
Reputation points Microsoft Vendor
0 answers
Syntax Error When Attempting to run PREDICT function against table in dedicated pool in Synapse Studio
I am attempting to complete the tutorial listed here : https://learn.microsoft.com/en-us/azure/synapse-analytics/machine-learning/tutorial-sql-pool-model-scoring-wizard I am able to complete all of steps until I get to the final step, where I use Synapse…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 2%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
1 answer
Federated Identity credentials auth in azure table sdk
Hi I was trying to use FIC creds for table sdk but was getting the following issue: CredentialUnavailableError: ManagedIdentityCredential authentication unavailable, no response from the IMDS endpoint. This is the doc I am following for FIC…
asked 2024-06-05T10:15:13.1933333+00:00
SanaCMSFT-8922
301
Reputation points
answered 2024-06-07T14:09:57.8+00:00
Amira Bedhiafi
17,616
Reputation points
1 answer
Azure Data Bricks - User Doesn't have permission to perform this action while connecting to Azure Synapse Dedicate Pool
We are connecting Azure Synapse Analytics - Dedicated Pool using the PySpark Code that runs from Azure Data Bricks using SQL Authentication. While running, we are getting the below error when we use a user with db_datawriter and db_datareader…
Azure
Azure
A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.
1,041 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,524 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,005 questions
asked 2024-06-07T11:26:00.4933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 97%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Praveen Sreeram
1
Reputation point
answered 2024-06-07T13:28:42.3466667+00:00
Vinodh247-1375
11,966
Reputation points
2 answers
Error using model.fit() in Synapse Apache Spark notebook
I have an unpredictable error. com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input at [Source: (String)""; line: 1, column: 0] I would like to train a time series model using SARIMAX with four…
asked 2023-07-17T14:39:07.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(195.2, 54%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAK%3C/text%3E%3C/svg%3E)
Abdelmonim Klaii
15
Reputation points
commented 2024-06-07T11:49:02.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 67%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Sebastian Bönig
0
Reputation points
1 answer
Synchronizing a Synapse table from an on-premise Oracle table
Hi, I've a Synapse table on a dedicated SQL pool and I need to update it reading from an Oracle table. I've already implemented a Self-hosted IR and I've already read from the Oracle table rightly with the Oracle connector. The requirement from my…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 41%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
How to Create Delta Table in Azure Synapse Analytics with Id Auto Increment Identity Column ?
I have created the Delta Lake Delta tables In ADLS using Synapse Notebook and in that table, I want to add an identity column (Auto increment 1,1) but I am not able to create the same, Below is my Create table script and error which i am facing. Table…
asked 2024-06-04T06:11:14.4766667+00:00
Vedant Desai
651
Reputation points
accepted 2024-06-07T09:52:14.62+00:00
Vedant Desai
651
Reputation points
0 answers
Enable Managed Private EndPoint for existing synapse resource
Hi all - We're seeking to activate MPN for an existing Synapse resource, but we're unable to locate the option via the UI. Please refer to the attached screenshot. Could you kindly advise if there are any available options to enable MPN for the existing…
asked 2024-06-05T23:47:08.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 73%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Abhiram Duvvuru
191
Reputation points Microsoft Employee
edited a comment 2024-06-07T09:41:56.95+00:00
Harishga
4,640
Reputation points Microsoft Vendor
0 answers
ArtifactDeletionTask status: 403; status message: Forbidden
Hi, We have a synapse pipeline that is used to copy data to cosmos DB and using the Synapse workspace deployment task in Azure DevOps to migrate from Dev to QA. While migrating we are getting error as below: 2024-05-05T21:30:06.5454516Z Getting Artifacts…
asked 2024-06-05T23:19:10.4066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 72%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Ajay Rana
0
Reputation points
commented 2024-06-07T09:33:22.4466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
11,290
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
How to grant Azure Synapse notebook manual run access to Azure ADSL Gen2 datalake?
Hello, I am experiencing an "Access unauthorized on the provided ADLS Gen2 account." error when I try to run a notebook manually from Azure Synapse workspace. However, the same notebook runs successfully when executed from a Synapse pipeline.…
asked 2024-01-17T13:43:20.78+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 91%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Dom-Su BoChum
20
Reputation points
accepted 2024-06-07T09:09:38.25+00:00
Dom-Su BoChum
20
Reputation points
1 answer
How would I create a role to be Synapse Admin but block particular pipeline and linked services
We a introducing a new source of data into Synapse which is highly sensitive. However currently my team have admin on Synapse and dedicated SQL pool. How can I allow them to keep some of the admin access but not allow them to see the pipelines and linked…
asked 2024-06-06T10:59:15.0466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 96%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
JayZee64
0
Reputation points
edited a comment 2024-06-07T08:10:26.4033333+00:00
Smaran Thoomu
11,290
Reputation points Microsoft Vendor
0 answers
How to use base parameter within a Synapse Notebook Pipeline activity?
I'm trying to use a base parameter (parameter_test )that I have for a notebook activity and reference that parameter within my notebook itself. I'm using pyspark, and need to pass in this parameter and set it to a variable within my notebook as shown…
asked 2024-06-06T19:32:57.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 88%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMM%3C/text%3E%3C/svg%3E)
Meghan Munagala
0
Reputation points
commented 2024-06-07T05:29:44.2066667+00:00
Smaran Thoomu
11,290
Reputation points Microsoft Vendor
0 answers
I can't do multiple unroll in flatten activity synapse dataflow
It was working quite well before, And started failing with "none.get" which is no where helping me to find the root cause, Even though I figured out that multiple flatten Unroll by cause the issue. My source is multiple Nested JSON try to…
asked 2024-06-06T16:34:28.0266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
Venkatesh S
0
Reputation points
commented 2024-06-07T05:26:46.3133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
7,190
Reputation points Microsoft Vendor
0 answers
Rows missing in dataflow output
I've noticed that around 3,000 rows go missing when running a dataflow that ingests data, performs tests and corrections, and sinks it into a new table. I use a filter to determine which rows need correction and which ones don't. However, when I tried…
asked 2024-06-04T13:15:44.6833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 3%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELM%3C/text%3E%3C/svg%3E)
Lepine-Mathieu, Guillaume
0
Reputation points
commented 2024-06-06T17:04:29.3933333+00:00
phemanth
7,190
Reputation points Microsoft Vendor
1 answer
Issue with Copy Dataverse data into Azure SQL using Synapse Link Template
I am experiencing issues using ADF I am trying to use the template "Copy Dataverse data into Azure SQL using Synapse Link" to move f&o data from datalake to Azure…
asked 2024-05-30T07:08:10.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 20%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDO%3C/text%3E%3C/svg%3E)
Dipo Onibile
0
Reputation points
edited a comment 2024-06-06T17:03:07.0733333+00:00
phemanth
7,190
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Setting up a Single-Node Cluster in Synapse
Could you let me know if it's possible to set up a single-node cluster using Azure Synapse Spark pool? My requirement is to avoid using any executor or worker nodes. I need a setup that only includes the master node. Is this configuration achievable?
asked 2024-06-06T11:36:33.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 15%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV0%3C/text%3E%3C/svg%3E)
vikranth-0706
140
Reputation points
accepted 2024-06-06T11:51:48.82+00:00
vikranth-0706
140
Reputation points
0 answers
We are not getting data in azure data lake Service v2 storage's container.
Hello, We are getting data in the Azure event hub sink through the POSTGRES SQL database but we are not getting files in the container which we have configured in an ADLS GEN 2 and while configuring we have an encounter that we are using a free…
asked 2024-05-30T07:02:45.27+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 90%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED%3C/text%3E%3C/svg%3E)
DNT
0
Reputation points
commented 2024-06-06T07:08:00.7066667+00:00
PRADEEPCHEEKATLA-MSFT
81,396
Reputation points Microsoft Employee